This invention relates to a method for stereoscopic viewing and imaging. More particularly, the invention is directed to a stereoscopic method for enhancing and displaying data from multiple sensors so that an observer can rapidly distinguish targets in a cluttered and noisy environment using the preattentive processing capabilities of the human visual system.
To be successful in combat, a pilot must rapidly integrate and synthesize a great deal of information in a noisy, confusing, and frequently disorienting environment. The soldier on the battlefield is increasingly provided with sophisticated electronic systems that rely on data acquired from a number of different sensors. Such electronic sensor systems are capable of intercepting electromagnetic radiation in frequency spectrums typically occupied by offensive and defensive military weapon systems as well as receiving signals in the visible, infrared and ultraviolet light range, and sensing pressure, vibration, sound and magnetic field variations. Significant advances have been made in target detection by processing and fusion of data received from multiple sensors. Recent efforts have explored cooperative control among various platforms and methods to obtain multi-asset synchronization of data. Research also has explored adaptive sensor management techniques to respond to situational changes, optimizing data distinctiveness, defining levels of responsibility, including sensor complement optimization, database information management, model based reasoning, context sensitivity and efficiency, and client/server relationships that take advantage of object oriented software and parallel processing techniques to achieve greater processing speed and thus make better real time use of the synergistic information obtained from multiple sensors. While such advances have improved target tracking and detection, true automatic target detection and tracking have not yet been achieved with any reasonable degree of confidence and, thus, the human operator is still very much in the control loop. Yet, comparatively little effort has gone into taking best advantage of the built in processing capabilities of the human perception system.
Efforts have been devoted instead, to extending the human visual perception system. This includes overlaying grayscale imagery from single or dual band infrared sensors and other weather penetrating sensor imagery with low-light visible imagery. For example, FLIR Systems Inc. (FSI) has developed a gray-scale pilot""s heads up display which overlays registered grayscale infrared and millimeter energy with the normal nighttime visual scene in order to provide a pilot with better visual reference during bad weather approaches. However, because different physical features are imaged by the various sensors such grayscale detection methods often actually result in degradation in object recognition and/or image quality. Aguilar, et al. at Massachusetts Institute of Technology (MIT) Lincoln Laboratory have experimented with a system for enhancing night vision based on the fusion of low-light visible and infrared sensors into a single color composite. xe2x80x9cField Evaluation of Dual-Band Fusion for Color Night Vision,xe2x80x9d M. Aguilar et al., SPIE Conference on Enhanced and Synthetic Vision, April 1999, SPIE Vol. 3691. Aguilar et al.""s approach assigns red, green and blue (RGB) colors to sensor channels so that the image uses brightness contrast to present information from the visible bands while using color contrast to represent the thermal vs. visible information in the scene. While the resultant false color images are more natural in appearance than grayscale night vision images (i.e., the sky is blue and the trees green) such attempts to color match different grayscale imagery from spectral regions outside of the visible band to the individual color cone response do not significantly enhance target recognition, especially in cluttered backgrounds.
It is an object of the invention to provide improved methods of stereoscopic viewing and imaging. It is a further object of the invention to provide methods for enhancing and displaying data from multiple sensors so that an observer can rapidly distinguish targets in a cluttered and noisy environment using the preattentive processing capabilities of the human visual system. These and other objects are satisfied, at least in part, by a method for processing signals from a plurality of sensors representing different spectral views of the same three dimensional scene that includes obtaining first left and right stereo signals from a pair of left and right sensors sensitive in a first spectral range which includes assigning a first Red-Green-Blue (RGB) color code to the first left and right stereo signals to enhance object recognition and perception, obtaining second left and right stereo signals from a pair of left and right sensors sensitive in a second spectral range, assigning a second Red-Green-Blue (RGB) color code to the second left and right stereo signals to enhance object recognition and perception. In a preferred embodiment of the invention, the first left and right sensor pairs are sensitive in the infrared range. In another preferred embodiment, the second left and right sensor pairs are sensitive in the visible range. In yet another embodiment, the method includes obtaining signals from third left and right sensor pairs sensitive in a third spectral range different from the first and second spectral ranges, and assigning a third Red-Green-Blue (RGB) color code to the third left and right stereo signals to enhance object recognition and perception. In another embodiment, the first, second and third left and right sensor pairs are sensitive in the near, mid and far infrared ranges, respectively.
Still another embodiment includes a method of displaying fused sensory data as color stereo images, the data obtained from stereo pair sensors, including assigning a Red-Green-Blue (RGB) color code to the first left and right stereo signals to enhance object recognition and perception, obtaining second left and right stereo signals from a pair of left and right sensors sensitive in a second spectral range, assigning a second Red-Green-Blue (RGB) color code to the second left and right stereo signals to enhance object recognition and perception and displaying the RGB color-coded data in a color stereoscopic image. In still yet another embodiment, the invention includes overlaying the RGB coded signals on a heads-up display so that periodically the RGB sensor fused information ramps in overall intensity from zero to a maximum percentage. The maximum percentage is determined for the particular application. Still other objects of the present invention will become readily apparent to those skilled in this art from the following description wherein, where there is shown and described a preferred embodiment of this invention, simply by way of illustration one of the modes to best carry out the invention. As it will be realized, the invention is capable of other different embodiments and its several details are capable of modification in various, obvious aspects all without departing from the invention. Accordingly, the drawing and descriptions will be regarded as illustrative in nature and not restrictive.